Stationary equilibria in stochastic games: structure, selection, and computation

This paper is the rst to introduce an algorithm to compute stationary equilibria in stochastic games, and shows convergence of the algorithm for almost all such games. Moreover, since in general the number of stationary equilibria is overwhelming, we pay attention to the issue of equilibrium selection. We do this by extending the linear tracing procedure to the class of stochastic games, called the stochastic tracing procedure. From a computational point of view, the class of stochastic games possesses substantial di culties compared to normal form games. Apart from technical di culties, there are also conceptual di culties, for instance the question how to extend the linear tracing procedure to the environment of stochastic games. We prove that there is a generic subclass of the class of stochastic games for which the stochastic tracing procedure is a compact one-dimensional piecewise di erentiable manifold with boundary. Furthermore, we prove that the stochastic tracing procedure generates a unique path leading from any exogenously speci ed prior belief, to a stationary equilibrium. A well-chosen transformation of variables is used to formulate an everywhere di erentiable homotopy function, whose zeros describe the (unique) path generated by the stochastic tracing procedure. Because of di erentiability we are able to follow this path using standard path-following techniques. This yields a globally convergent algorithm that is easily and robustly implemented on a computer using existing software routines. As a by-product of our results, we extend a recent result on the generic niteness of stationary equilibria in stochastic games to oddness of equilibria. JEL classi cation: C63, C72, C73